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Abstract. We study a version of the stochastic "tug-of-war" game, 
played on graphs and smooth domains, with the empty set of terminal 
states. We prove that, when the running payoff function is shifted by 
an appropriate constant, the values of the game after n steps converge 
in the continuous case and the case of finite graphs with loops. Using 
this we prove the existence of solutions to the infinity Laplace equation 
with vanishing Neumann boundary condition. 

1. Introduction 

1.1. Background and previous results. For a (possibly infinite) graph 
G = {y,E), the stochastic tug-of-war game, as introduced in [9], is a two- 
player zero-sum game defined as follows. At the beginning there is a token 
located at a vertex x . At each step of the game players toss a fair coin 
and the winning player gets to move the token to an arbitrary neighbor of x. 
At the same time Player II pays Player I the value /(x), where / : 1/ — )• M is 
a given function on the set of vertices, called the running payoff. The game 
stops when the token reaches any vertex in a given set W G V, called the 
terminal set. If y G W is the final position of the token, then Player II pays 
Player I a value of g{y) for a given function g: W called the terminal 
payoff. One can show that, when g is bounded and either / = 0, inf / > or 
sup / < 0, this game has a value (Theorem 1.2 in [9j), which corresponds to 
the expected total amount that Player II pays to Player I when both players 
"play optimally". 

Among other reasons, these games are interesting because of a connec- 
tion between the game values and viscosity solutions of the infinity Laplace 
equation. Let C M*^ be a domain (open, bounded and connected set) with 
boundary dCl, and let /: — t- M and g: dCl — t- M be continuous func- 
tions. Define the graph with the vertex set so that two points x,y G il. 
are connected by an edge if and only if the intrinsic path distance between 
X and y in is less than e. Playing the game on this graph corresponds to 
moving the token from a position x G to anywhere inside the ball with the 
center in x and radius e, defined with respect to the intrinsic path metric 
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in Q. Consider this game with the running payoff e^/, the terminal set d^l 
and the terminal payoff g. By Dynamic programming principle if the value 
of this game exists then it is a solution to the finite difference equation 

u(x) I min u + max u] = f(x), 

^ ' 2 \B{x,e) B{x,e) J ^ ^' 

for all X £ Q, and u{y) = g{y), for all y G dQ. In [9] it was shown that, 
under certain assumptions on the payoff function /, the game values with 
step size e converge as e converges to zero appropriately. Moreover the limit 
u is shown to be a viscosity solution to the non-linear partial differential 
equation 

J - AooU = f in il, 

\u = g on dQ. 

Using finite difference approach and avoiding probabilistic arguments, Arm- 
strong and Smart [1] obtained general existence results for this equation and 
the uniqueness for typical shifts of the function /. Several modifications of 
this game have also been studied, including using biased coins, which cor- 
responds to adding a gradient term to the equation (see [8]) and taking 
the terminal set to be a non-empty subset of d^l, which corresponds 
to Dirichlet boundary conditions on W and vanishing Neumann boundary 
conditions on dO, \ W (see [2] , [3] and [5] ) . 

A crucial property of these games is the fact that the terminal set is non- 
empty which ensures that the game can stop in finite time. However to use 
the above connection in order to study the infinity Laplace equation with 
pure vanishing Neumann boundary conditions, one would have to consider 
this game without the terminal set. This is the content of this paper. 

In the following two subsections we introduce the notation and give nec- 
essary definitions. In Section [2] we study the tug-of-war games of finite 
horizons (defined below) on finite graphs. The results we obtain are used 
in Section [3] to prove the existence of solutions of infinity Laplace equations 
with pure vanishing Neumann boundary conditions. Section S] contains a 
discussion about uniqueness. 

1.2. Setting and notation. All graphs that we consider in the text will 
be connected and of finite diameter (in graph metric), but we allow graphs 
to have an uncountable number of vertices (and vertices with uncountable 
degrees). However, for most of the main results we will need additional 
assumptions. 

Definition 1.1. Let {V,d) be a compact length space, that is, a compact 
metric space such that for all x,y G V the distance d{x, y) is the infimum 
of the lengths of rectifiable paths between x and y. For a fixed e > 0, the 
€- adjacency graph is defined as a graph with the vertex set V, such that two 
vertices x and y are connected if and only if d{x, y) < e. When the value of 
e is not important we will simply use the term adjacency graph. 
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A particular example of e-adjacency graphs, already described in the pre- 
vious subsection, corresponds to taking {V, d) to be a closure of a domain 

C M'^ with boundary dO. and the intrinsic metric in Q.. This means 
that y) is equal to the infimum of the lengths of rectifiable paths con- 
tained in Q, between points x and y. We will call these graphs Euclidean 
e-adjacency graphs. Note that in the present work we will limit our atten- 
tion to domains with boundary (meaning that for any x G dO. we can 
find open sets U C M"^"*^ and V C M" containing and x respectively, a 
function cj): U ^ M"~^ and an isometry of M" which maps (0, 0(0)) to x and 
the graph of onto V n dO) . 

While the fact that (Q, d) is a metric space is fairly standard, at this point 
we need to argue that this space is compact. Actually one can see that the 
topology induced by this metric space is the same as the Euclidean topology, 
and we only need to show that it is finer. To end this assume that {xn) is a 
sequence of points such that lim„ |x„ — x\ =0, for some x G 0. If x G it is 
clear that lim„ d{xn,x) = 0. On the other hand if x G then let yn be the 
closest point on to x„. It is clear that d{xn,yn) = \xn — Unl converges 
to zero as n — )• oo and lim„ \x — yn\ = 0- To complete the argument simply 
consider the paths between x and ?/„ contained in dQ, which are obtained 
by composing a parametrization of dQ in a neighborhood of x and an 
affine function. The lengths of these paths converge to zero. 

Another interesting class of graphs will be finite graphs with loops at each 
vertex. Intuitively, these graphs might be thought of as e-adjacency graphs 
where {V, d) is a finite metric space with integer valued lengths and e = 1. 

Note that for two vertices x and y we write x ~ y if x and y are connected 
with an edge. The graph metric between vertices x and y will be denoted 
by dist(x,y) in order to distinguish it from the metric in Definition 11.11 
The diameter of a graph G will be denoted by diam(G). We consider the 
supremum norm on the space of functions on V, that is ||ii|| = max^; |n(x)|. 
When V is compact length space, this norm makes C(y, M), the space of 
continuous functions on V , a Banach space. We also use the notation 
B{x,e) = {y £ V : d{x,y) < e} (we want to emphasize that, in contrast 
with [9], the balls B{x,e) are defined to be closed) . 

We consider a version of the tug-of-war game in which the terminal set 
is empty, but in which the game is stopped after n steps. We say that this 
game has horizon n. At each step, if the token is at the vertex x Player II 
pays Player I value /(x). If the final position of the token is a vertex y then 
at the end Player II pays Player I value g{y). Here / and g are real bounded 
functions on the set of vertices called the running and the terminal payoff. 
Actually this game can be realized as the original stochastic tug-of-war game 
introduced in [9] played on the graph Gx{l,2,...,?7,} (the edges connecting 
vertices of the form (f , i) and (w, i + 1), where v and w are neighbors in G), 
for the running payoff f{v,i) := f{v), the terminal set V x {n} and the 
terminal payoff g{v,n) := g{v). Note that the same game with vanishing 
running payoff has appeared in a different context in [3] . 
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Define strategy of a player to be a function that, for each 1 < k < 
n, at the A;-th step maps the previous k positions and k coin tosses to 
a vertex of the graph which neighbors the current position of the token. 
For a Player I strategy Si and Player II strategy Su define Fn{Si,Sii) as 
the expected payoff in the game of horizon n, when Players I and II play 
according to strategies Si and Su respectively. Define the value for Player 
I as ui^n = sup^j inf^jj 5n), and the value for Player II as un^„ = 

infsjj sup5j Fn{Si, Su). Note that we consider both uj ^ and Mil n as functions 
of the initial position of the token. Intuitively ui^n{x) is the supremum of 
the values that Player I can ensure to earn and uii,nix) is the infimum of 
the values Player II can ensure not to overpay, both in the game of horizon 
n that starts from x €V. It is clear that ui^ = uu^ = g and one can easily 
check that ni^„ < uu^n- In a game of horizon n+1 that starts at x, if Players 
I and II play according to the strategies Si and Su which, in the first step, 
push the token to xi and xu respectively, we have 

Fn+l{Sl,Sll){x) = /(x) + ^(F2,„+i(cSi,5ii)(xi)+F2,„+i(5i,cSii)(xii)), (1) 

where F2^n+i{Si, Sii){y) is the expected payoff between steps 2 and n + 1 
conditioned on having position y in the second step of the game. It is easy 
to see that using the above notation ui^n = sup^^ inf^jj F2^„+i(5i, and 
un,n = inf^jj sup^j F2^„+i(5i, where Si and 5ii are strategies for Player 
I and Player II in the game that lasts for n + 1 steps. Now using induction 
in n and ([T|) one can check that Un ■= ui^n = uu,n for any n, and that the 
sequence satisfies uq = g and 

Un+i{x) = l-(minun{y) +maxn„(y)) + fix). (2) 

2 \ y^x y^x / 

Furthermore the infima and the suprema in the definitions of ui^n and nn,n 
are achieved for the strategies that at step k of the game of horizon n pull the 
token to a neighbor that maximizes (minimizes) the value of Un-k (such a 
neighbor exists for finite degree graphs, and also in the case of an e- adjacency 
graph provided Un-k is known a priori to be continuous). 

In this paper we will mainly study the described game through the recur- 
sion 

Remark 1.2. In the case of e-adjacency graphs we will normally assume 
that the terminal and the running payoff are continuous functions on V and 
heavily use the fact that the game values Un are continuous functions. To 
justify this it is enough to show that, if n is a continuous function on V, 
then so are t<*(x) = max^j-^ u and Uei^'^) — ^^^B{x,e) ^- this, one only 
needs to observe that for any two points x, y G y such that d{x, y) < 5, any 
point in B{x, e) is within distance of 5 from some point in B{y, e), and vice 
versa. Now the (uniform) continuity of and Ue follows from the uniform 
continuity of u, which holds by compactness of V. 
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For a game played on an arbitrary connected graph of finite diameter, 
if the sequence of game values (un) converges pointwise, the limit u is a 
solution to the equation 

u{x) — -(minM(y) + maxu(y)) = f{x). (3) 
2 y-^x y^x 

The discrete infinity Laplacian A^oU is defined at a vertex x as the negative 
left hand side of the above equation. As mentioned before, the case of Eu- 
clidean e-adjacency graphs is interesting because of the connection between 
the game values and the viscosity solutions of ([5]) defined in Definition 11.41 
To observe this it is necessary to scale the payoff function by the factor of e^. 
Therefore in the case of e-adjacency graphs we define the e-discrete Laplace 
operator as 

A e / N (min G£,(^ ) u{y) + max ^^(^ ) u{y)) - 2u{x) 

^ooU{x) = '-^ ^2 ' 

and consider the equation 

-A^n = /. (4) 

Remark 1.3. Observe that, compared to the discrete infinity Laplacian, we 
removed a factor 2 from the denominator. This definition is more natural 
when considering the infinity Laplacian Aqo described below. As a conse- 
quence we have that the pointwise limit u of the game values played 
on an e-adjacency graph with the payoff function e^//2 is a solution to dH). 

We will consider the infinity Laplace equation on a connected domain 
with boundary 517, with vanishing Neumann boundary conditions 

- AooM = / in r2, 

V^u = on dn. 

Here z/ (or more precisely denotes the normal vector to di} at a point 

X G do,. The infinity Laplacian A^o is formally defined as the second deriv- 
ative in the gradient direction, that is 

AooU = \Vu\''^'^Ux^Ux^xjU^y (6) 

We will define the solutions of ([5]) and prove the existence in the following 
viscosity sense. First define the operators and A^ as follows. For a 
twice differentiable function u and a point x such that V'u(x) ^ define 
A+ti and A^u to be given by ([6]), that is A'^u{x) = A^u{x) = Aoou{x). 
For x such that Vu{x) = define A+u(x) = max{Y^ ■ j UxiXj{x)'ViVj} and 
A^u{x) = mm{Y^- J UxiXjix)viVj} , where the maximum and the minimum 
are taken over all vectors v = (vi, . . . , v^) of Euclidean norm 1. 

Definition 1.4. A continuous function u: — t- M is said to be a (viscosity) 
subsolution to (0) if for any function G C°°(J7) (infinitely differentiable 
function on an open set containing $7) and a point xq G Q, such that u — 
has a strict local maximum at xq, we have either 
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a) -^toHxo) < f{xo) or 

b) xo G dfl and Vj,(^.„)(/?(xo) < 0. 

A continuous function u: $7 — )• M is said to be a (viscosity) supersolution if 
—u is a (viscosity) subsolution when / is replaced by — / in dS]). A continuous 
function u is said to be a (viscosity) solution to ([5]) if it is both a subsolution 
and a supersolution. 

Note that the notion of (sub, super)solutions does not change if one re- 
places the condition ip G C°°{Q) with 93 E C^(J7). 

Remark 1.5. The above definition, while having the advantage of being 
closed under taking limits of sequences of solutions, might be slightly unnat- 
ural because the condition in a) is sufficient for xq G dil. at which u — ip has 
a strict local maximum. Following [6j we can define the strong (viscosity) 
subsolution as a continuous function u such that, for any ip G C°°{Q) and 
any xq G 0,, at which u — (p has a strict local maximum, we have 

a') -A+ V9(xo) < fixo), if xq G Q, 
b') Vuiplxo) < 0, if Xo G 

Strong (viscosity) supersolutions and solutions are defined analogously. While 
it is clear that the requirements in this definition are stronger than those in 
Definition 11.41 it can be shown that, when is a convex domain, any (sub, 
super) solution is also a strong (sub, super)solution. To show this assume 
that n is a viscosity subsolution to 1^ in the sense of Definition 11.41 and 
let X G dQ and <p G C°°{Q) be such that u — ip has a strict local maxi- 
mum at X and V^^ix) > 0. Without the loss of generality we can assume 
that X = and that the normal vector to di} at is = — e^, where 
is the d-th. coordinate vector. Thus by the convexity, the domain lies 
above the coordinate plane Xd = 0. Now define the function (p G C°°(0) as 
(f){y) = (f{y)+ayd- /3{yd)'^ , for positive a and /3. Since V(/)(0) = V(p{0)+aed, 
for a small enough we still have V,y0(O) > 0. Moreover the Hessian matrix 
of (j) is the same as that of ip, with the exception of the (d, d)-entry which is 
decreased by 2(3. Since > and V;^0(O) does not depend on /3, for /3 

large enough we will have — Aj^0(O) > /(O). Moreover since we can find an 
open set U such that ip{y) < (j){y) for all y £ QCiU, we have that u — cp has 
again a strict local maximum at 0. Since it doesn't satisfy the conditions in 
Definition II. 4t this leads to a contradiction. 

1.3. Statements of results. We want to study the values of games as their 
horizons tend to infinity. Clearly taking payoff function / to be of constant 
sign will make the game values diverge. Since increasing the payoff function 
by a constant c results in the increase of the value of the game of horizon 
n by nc, the most we can expect is that we can find a (necessarily unique) 
shift / -|- c of the payoff function / for which the game values behave nicely. 
The first result in this direction is the following theorem which holds for all 
connected graphs of finite diameter. 
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Theorem 1.6. For any connected graph G = iV^E) of finite diameter and 
any hounded function / : y — )• M there is a constant Cf, such that the fol- 
lowing holds: For any bounded function uq: V ^ M., if (un) is the sequence 
of game values with the terminal and running payoffs uq and f respectively, 
then the sequence of functions {un — ncf) is bounded. 

We will call c/ from Theorem 11.61 the Player I's long term advantage for 
function /. For both adjacency graphs and finite graphs with loops we have 
convergence of the game values. 

Theorem 1.7. Let G = {V,E) be either an adjacency graph, or a finite 
graph with a loop at each vertex. Let f,UQ: F — )• M be functions on the set 
of vertices, which are assumed to be continuous if G is an adjacency graph. 
Assume Cf = 0. In a game played on G with the terminal and running 
payoffs Uq and f respectively, the sequence of game values {un) is uniformly 
convergent. 

The following theorem gives the correspondence between the tug-of-war 
games and the equation ([3]). While for adjacency graphs and finite graphs 
with loops this is a straightforward corollary of Theorem 11.71 this result also 
holds for all finite graphs, even when Theorem 11.71 may fail to hold (see 
Example EJ]). 

Theorem 1.8. Let G = {V,E) be either an adjacency graph, or a finite 
graph. Let f,UQ:V — t- M be functions on the set of vertices, which are 
assumed to be continuous if G is an adjacency graph. Then the equation ([3]) 
has a solution u if and only if cj = 0. 

Let y be a compact length space and a consider a function / € C{V). 
To emphasize the dependence on e define cj(e) as the Player I's long term 
advantage for a game played on the e-adjacency graph defined on V with 
the running payoff /. As already mentioned, to study the limiting case e J, 
for e-adjacency graphs, we need to scale the running payoff function by a 
factor of e^, that is, the we take e^f as the running payoff function. Note 
that the Player I's long term advantage corresponding to this game is equal 
to e^Cfie). 

The first problem one encounters is the fact that c/(e) depends on the 
value of e (see Example 13. Ih . The following theorem gives meaning to the 
notion of Player I's long term advantage in the continuous case. 

Theorem 1.9. For any compact length space V and any continuous function 
/: y — )■ M the limit lim^^o c/(e) exists. 

We will denote the limit from the above theorem by Cf = liui^io c/(e)- 

Theorem 1.10. Let V be a compact length space, and (e„) a sequence of 
positive real numbers converging to zero. Any sequence (un) of continuous 
functions on V satisfying — A^n^ = f — Cf{en) and such that is in the 
range of Un for all n, has a subsequence converging to a Lipshitz continuous 
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function. Moreover the Lipshitz constant is bounded by a universal constant 
multiple o/diam(y) 11/11 . 

For Euclidean e-adjacency graphs, the hmits from Theorem 11.101 give us 
viscosity solutions of (j5]). 

Theorem 1.11. Let be a domain of finite diameter with boundary dQ 
and /: — 7- M a continuous function, such that Cf = 0. Then the equation 
([5|) has a viscosity solution u which is Lipshitz continuous, with Lipshitz 
constant depending on Q and the norm ||/||. 

It is natural to expect the existence of viscosity solutions to ([5]) only for 
one shift of the function /. This is proven in the following theorem for 
convex domains 0. 

Theorem 1.12. LetQ, be a convex domain of finite diameter with bound- 
ary do, and / : i7 — )• M a continuous function. Then the equation (0) has a 
viscosity solution u if and only ifcf = 0. 

Remark 1.13. Directly from Theorem 11.61 one can deduce that cxf = Xcf 
and Cf^x = Cf + X for any A G M. For compact length spaces, after taking 
an appropriate limit, we obtain the same properties for Cf. Thus Theorem 
11.81 tells us that, under its assumptions, for any function g on the vertex set 
(continuous in the case of adjacency graphs), there is a unique constant c, 
such that equation ([3]) can be solved when f = g — c. Theorems 11.111 and 
11.121 tell us that any function g G C{Q) can be shifted to obtain a function 
/ G C(r2) for which ([5]) can be solved, and that this shift is unique when 
is convex. 

Remark 1.14. In the case of finite graphs and for a fixed / (such that Cf = 0), 
the solutions to the equation ([3]) are not necessarily unique (even in the case 
of finite graphs with self loops). A counterexample and a discussion about 
the continuous case is given in Section [H 

2. The discrete case 

Since the terminal payoff can be understood as the value of the game of 
horizon 0, we will not explicitly mention the terminal payoff when it is clear 
from the context. 

Lemma 2.1. Let G = {V,E) be a connected graph of finite diameter and 
let f , g, no and vq be bounded functions on V. 

(i) Let (un) and {v„) be the sequences of values of games played with 
the running payoff f. If uq < vq, then Un < Vn for all n > 0. 
Furthermore, if for some c G M we have vq = uq + c, then Vn = Un + c 
for all n > 0. 

(ii) Let {u\) and (n^) be sequences of values of games played with the 
terminal payoffs Uq = Uq = uq and the running payoffs f and g 
respectively. If f < g then u\ < n^, for all n. Furthermore if for 
some c G M we have g = f + c, then = + nc for all n > 0. 
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Proof. All statements are easy to verify by induction on n using relation 

m- □ 

Lemma 2.2. For a connected graph G = {V, E) of finite diameter and 
hounded functions f,UQ:V — t- M, let {un)n be the sequence of values of 
games played on G with running payoff f . Then for all n >0 we have 

maxu„ — minu„ < (maxuo — minuo) + diam(G')^(max / — min/). 

Proof. Consider the sequence of game values (vn) played with the running 
payoff / and zero terminal payoff. From part (i) of Lemma 12.11 we get 
Vn + minuo < Un < + maxuo- This implies that 

max Un — minn„ < (maxt;„ — minu„) + (maxuo — minuo). 

From this it's clear that it is enough to prove the claim when uq = 0. 
Furthermore, by part (ii) of Lemma 12.11 it is enough to prove the claim for 
an arbitrary shift of the payoff function /, and therefore we assume that 
min/ = 0. This implies that m„ > 0, for n > 0. Now, for a fixed n, by 
Lemma |2. II (i), playing the game of horizon n — k with the running payoff 
/ and the terminal payoffs and 0, gives the game values Un and Un-k 
respectively, and 

Un^k < Un- (7) 

Fix a vertex z ^ V . For a vertex y & V pick a neighbor z(y) G V of y 
so that dist{z{y), z) = dist{y,z) — 1 (if y and z are neighbors then clearly 
z{y) = z). Let ^{'j ^ be the optimal strategy for Player II in the game of 
horizon k. For any Player I strategy Si^k for a game of horizon fc, we have 
Fk{S\,k,Sii j^) < Uk. Now define the "pull towards z" strategy Su for a game 
of length n as follows. At any step of the game if the token is at the vertex 
y z and if z is not among the past positions of the token, then strategy Su 
takes the token to the vertex z{y). If T is the first time at which the token 
is at the vertex z, at this point Player II starts playing using the strategy 
•Siin-T- -'-f -^t is the position of the token at time t, then it can be easily 
checked that for Yt = (diam(G) — dist{Xt, z))'^ — t, the process l^fAT is a 
submartingale, with uniformly bounded differences. Moreover the stopping 
time T has a finite expectation since it is bounded from above by the first 
time that Player II has won diam(G) consecutive coin tosses (partition coin 
tosses into consecutive blocks of length diam(G) and notice that the order 
of the first block in which Player II wins all the coin tosses has exponential 
distribution with mean 2^^^™('-')). Therefore applying the optional stopping 
theorem we get E{Yt) > E{Yo), hence E(r) < diam(G)2. Now consider the 
game in which Player I plays optimally and Player II plays according to 
the above defined strategy Su. Since each move in the optimal strategies 
depends only on the current position of the token, by the independence of 
the coin tosses, we have that conditioned on T = A;, the expected payoff 
in steps /c + 1 to n is bounded from above by Un-k{z). Clearly the total 
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payoff in the first k steps is bounded from above by fcmax/. For x 7^ z tlie 
strategy 5jj is suboptimal and 

n 

Unix) < P(r = k){k max / + Un-k{z)) + F(r > n)n max /. 

k=l 

Since / is a non-negative function, so is Un for any n. Using tliis witfi ([7]) 
we get 

n 

Unix) < ^P(r = k)kmaxf + Uniz) + P(r > n)n max/ 
k=l 

< n„(z) +E(r) max/. 

Since x and z are arbitrary and E(T) < diam(G)^ for all x and z, the claim 
follows. □ 

Proof of Theorem \1.6[ As in the proof of Lemma 12.21 we can assume that 
uq = 0. Denote M^, = maxuk and m^, = minufc. By part (i) of Lemma |2.H 
playing the game with the constant terminal payoff gives the sequence of 
game values (n„ + Mfc)„. Comparing this to the game with the terminal pay- 
off Ufc we obtain Un+kix) < Unix) + M^. Taking maximum over all vertices 
X leads to the subaditivity of the sequence (M„), that is Mn+k < + Affc- 
In the same way we can prove that the sequence (m„) is superaditive. By 
Lemma |2. 21 we can find a constant C so that M„ — m„ < C for any n, and 
thus we can define 

Mn . ,Mn 

Cf := lim = mf = lim = sup . (8j 

n n n n n ri n n 

Then, for any n > we have 

ncf < Mn < rUn + C < ncf + C, 
and therefore, for any x G V 

\unix) — ncf \ < max{|M„ — ncf\, \mn — ncf\} < C. 

□ 

For an arbitrary graph G = iV,E) and a function f on V, define the 
(non-linear) operator Af acting on the space of functions on V, so that for 
each x G V 

Afuix) = — ( maxu(y) + minn(y) ) -|- fix). 
2 V y^x y^x J 

Lemma 2.3. Assume G = (F, E) is either an adjacency graph or a finite 
graph. Let f , u and v be functions on V , which are also assumed to be 
continuous if G is an adjacency graph. Then we have 

min(t> — u) < miniAfV — Afu) < maxiAfV — Afu) < max(t> — u), (9) 

and 

\\AfV — Afu\\ < \\v — u\\. (10) 
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Moreover for X £ V we have Afv{x) — Afu{x) = max{v — u) if and only if for 
any two neighbors yi and y2 of x such that u{yi) = mmyr^x u{y), and v{y2) = 
TuoKyr^xviy), we also have v{yi) = Toiny^xviy), o-nd u{y2) = maxy^^ ii(y) 
and v{yi) — u{yi) = max(f — u), for i £ {1, 2}. 

Proof. Fix a vertex x G V and note that maxy^xviy) < maxy^xu{y) + 
max(u — u) and miny^xviu) < miny^j; + max(i; — u). Adding these 
inequaUties one obtains Afv{x) < Afu{x) + max(t; — u). The inequahty 
min(u — u) < mm{AfV — Afu) now follows by replacing u and v by —u 
and —V respectively, and ()10p follows directly from ([9]). It is clear that the 
equality Afv{x) — Afu{x) = max(T; — u) holds if and only if both 



maxf (y) = maxn(y) + max(?; — u) (11) 

y^x y^x 



and 



mint'(y) = mmu{y) + max(t' — u) (12) 

yr^x y^x 

hold. It is obvious that the conditions in the statement are sufficient for 
(fTTI) and (fT2]) to hold, and it is only left to be proven that these conditions 
are also necessary. To end this assume that both (llip and (I12p hold and 
take yi and y2 to be arbitrary neighbors of x such that u[yi) = min^^a; u{y) 
and v{y2) = max^^^; v{y). Clearly we have 

min?;(y) < v{yi) < u{yi) + m.ax{v — u) = minti(y) + max(i; — u), 

y^x yr^x 

and moreover all the inequalities in the above expression must be equalities. 
This implies both v{yi) = u{yi) + max(u — u) and v{yi) = minj,^^, v{y). The 
claim for y2 can be checked similarly. □ 

The following proposition proves Theorem 11.71 in the case of finite graphs 
with loops. For a sequence of game values {un)n with the running payoff /, 
define mI{uq) = max^jgy (ii„(x) - n„_i(x)) and ml,{uo) = mina;gi/(n„(x) - 

Un-lix)). 

Proposition 2.4. Under the assumptions of Theorem \1. 7| the sequence of 
game values converges if it has a convergent subsequence. 

Proof. If mI{uo) = —5 < for some n > 1, we have Un < Un^i—S and by ap- 
plying part (i) of Lemma [2. II we obtain Um < Um-i — (5, for any m > n. This 
is a contradiction with the assumption that Cf = 0. Therefore m/(wo) > 
and similarly mn{uo) < 0. By Lemma [2.31 the sequences (M/(no))n and 
{mn{uo))n are bounded and non- increasing and non-decreasing respectively 
and therefore they converge. 

Let w be the limit of a subsequence of {un)n- Assume for the moment 
that m/ {w) = m{{w) = 0, or equivalently Afvu = w. Now Lemma 
implies 

||Un+l — II = ll^/lin — ^/w|| < \\Un — w\\. 
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Therefore — is decreasing in n and, together with the fact that is 
its accumulation point, this yields lim„, \\un — w\\ = 0. Therefore it is enough 
to prove that m({w) = m{^{w) = 0. The rest of the proof will be dedicated 
to showing m( (w) = (the claim m{^{w) = following analogously). 

First we prove that {Mi{w)) is a constant sequence. Assume this is 
not the case, that is for some k we have Mj^_^-^^{w) < mI{w). For any 

n > 1 the mapping v i— )• mI{v) is continuous and therefore we can find a 
neighborhood U oi w iU C C{y,M) in the adjacency case) and (5 > such 
that mI_^^{v) < mI{v) - 5, for any v e U. Observing that M[{un) = 
M^_l_^(no), and that Un for infinitely many positive integers n, we have 
M^j^(tio) < M^{uo) — 6 for infinitely many positive integers £. This is 

a contradiction with the fact that (M/(no))n is a nonnegative decreasing 
sequence. 

Now let M = mI{w) > and denote by (wn) the sequence of game values 
with terminal and running payoffs wq = w and / respectively. Define the 
compact sets Vn = {x G V : w„+i(x) = 'Wn{x)+M} and t„ = miiixi^Vn "^nix). 
Taking x £ Vn we have M = Wn+i{x) — Wn{x) = max(t(;„ — Wn-i) and thus 
we can apply Lemma [2^ to find y ~ x such that Wn{y) = min^^a; Wn{z) and 
y G Vn-i- Because the graph G satisfies x ~ x for any vertex x, we obtain 

Wn{x) > Wn{y) = Wn-l{y) + M> + M, (13) 

for any x G V^. Taking the minimum over x G yields t„ > + M. 
For M > this is a contradiction with the boundedness of the sequence 
(wn)) which in turn follows from Cf = 0. Thus M = which proves the 
statement. □ 

Remark 2.5. Note that in the case of finite graphs, the first inequality in 
([T3l) is the only place where loops were used. 

The existence of accumulation points will follow from Lemma 12.71 which 
in turn will use the following lemma. Note that these two lemmas can replace 
the last paragraph in the proof of Lemma 12.41 However we will leave the 
proof of Lemma [2.41 as it is, since it gives a shorter proof of Theorem 11.71 for 
finite graphs with loops. 

Lemma 2.6. Under the assumption of Theorem \1.7\ we have 

lim(n„+i - Un) = 0. 

n 

Proof. We will prove that lim„ max{un+i — Un) = 0. The claim then follows 
from the fact that lim„ min(ti„_|_i — n„) = which follows by replacing n„ 
by -Un and / by -/. 

First assume that for some real numbers Ai and A2, a vertex x €z V and 
a positive integer n we have Un+i{x) — Un{x) > Xi and Mn(uo) < A2. Since 
maxz^x Un{z) — maxz^x Un-iiz) < A2, by ^ we see that 

minn„(z) — minn„_i(z) > 2Ai — A2. 
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This implies that for a vertex y ~ x, such that = mmz^xUn-i{z), 

we have 

mm{un{x),Un{y)} >minu„(z) > Un-i(y) + 2Ai - A2. (14) 

We win inductively apply this simple argument to prove the statement. 

As argued in the proof of Proposition 12.41 the sequence (m/ (uq)) is non- 
increasing and nonnegative and therefore converges to M = lim„M/(no) > 
0. For a fixed 5 > let no be an integer such that Mi{uo) < M + S, 
for all n > uq. For a given positive integer k, let xq be a point such that 
Uno+k{xo) — Un„+k-i{xo) > M. Then applying the reasoning that leads to 
(jl4p for Ai = M and A2 = M + 5, we can find a point xi such that 

min{u„o+fc_i(a;o),n„o+fc_i(a;i)} > u„,)+fc-2(a;i) + (M - 5). 

If A; > 3 we can apply the same argument for functions UnQj^k-i-, Uno+k-2 
and u„y+fc_3, point xi, Xi = M — 6 and X2 = M + 6. Inductively repeating 
this reasoning we obtain a sequence of points (x^), 1 < i < k — 1 such that 

inm{uno+k-e{xe-i),Uno+k-e{xi)} > Uno+k-e-i{xe) + M - (2^ - 1)6. 

Summing the inequalities 

Uno+k-dXi-l) > Uno+k-£-l{xi) + M - (2^ - 1)5, 

for 1 < £ < — 1 and UnQ+k{xo) — UnQ+k-i{xo) > M leads to 

Uno+k{xo) > Un„{Xk-l) + kM - 2^6, (15) 

for all A; > 1. Taking k{5) to be the smallest integer larger than we 
obtain 

Uno+k(5){xQ) > Uno{Xk{S)-l) + ^^(~^^~^^ ~ ' (^6) 

If M > then lim^^o ^^( ^"lig^^ - 2) =00, which by ^ implies that the 
sequence (u„) is unbounded. This is a contradiction with the assumption 
that c/ = 0. □ 

Lemma 2.7. The sequence of game values {un) for a game on an adjacency 
graph is an equicontinuous sequence of functions. 



The proof of this lemma uses an idea similar to the proof of Lemma [ 
We obtain a contradiction by constructing a sequence of points along which 
the function values will be unbounded. Since the induction step is more 
complicated we put it into a separate lemma. First define the oscillation of 
a continuous function v: F — )• M as osc(u,5) = suprf(2, ,^)<5 \v{x) — v{y)\. 

Lemma 2.8. Let (un) be a sequence of game values played on an adjacency 
graph. Assume that for positive real numbers Xi, X2, A3, p < e and a positive 
integer n we have 

osc(/,p) < Ai, osc{un,p) < A2, and Un < Un+l + A3. (17) 
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Let be a pair of points which satisfies d{x,y) < p and Un+i{x) — 

Un+i{y) > 5, for some 5 > 0. Then there are points (xi,yi) which satisfy 
d{xi,yi) < p, and the inequalities 

n„(xi)-n„(yi) >2(5-2Ai-A2, (18) 

and 

un+i{y) - un{yi) >25- 2Ai - A2 - A3. (19) 

Proof, li 5 < A1 + A2/2 then consider the set S = {z : Un{z) < Un+i{y) + X3}, 
which is nonempty by the last condition in (jl7p . If S is equal to the whole 
space V, then ([19]) will be satisfied automatically, and take xi and yi to 
be any points such that d{xi,yi) < p and Un{xi) > Un{yi) (so that (fT8|) 
is satisfied). Otherwise, since V is path connected we can choose points 
xi and yi so that d{xi,yi) < p and yi G S (so that ([19]) is satisfied) and 
xi ^ S (so that (|18p is satisfied). In the rest of the proof we will assume 
that 5 > Ai + A2/2. 

Choose points Xm, xm, Um, and yM so that 

Unixm) = ram;,^xUniz), Un{xM) = max^^^; n„ (z) , 
Un{ym) = min^^y Un{z), Uniyu) = max^^y Uniz). 

Take a point Zm such that d{y,Zm) < e — d{x,y) and d{zm,ym) < P, which 
surely exists, since d{x,y) < p and d{y,ym) < £• By the triangle inequality 
this point satisfies d(x, z^) < e. Therefore we have Zm ^ x, Zm ^ y and 

d(ym,{2: : z ~ ~ y}) < p. (20) 

Analogously we construct a point zm such that d{xM, zm) < P and d{zM,y) < 
e. Now we have 

Unixu) - Uniyu) < Un{xM) " 'Un(^M) < A2. (21) 

Next calculate 

{Un{xM) - Uniyu)) + { miu Un{z) - Un{ym)) 

n{Xm) Un iVm)) 

= 2{un+i{x) - f{x) - un+i{y) + f{y)) 

> 26 -2X1. (22) 

Plugging ([21]) into ([22]) we get 

min Uniz) - Un{ym) >26 - 2Xi - X2. (23) 

Now define r as the supremum of the values f such that for every zq E 
B{ym,r) we have n„(2;o) < mmz^x,z'^yUn{z). By ([20]) . ([23]) and the assump- 
tion on 5 it follows that r is well defined and < r < p. Finally we take 
a point xi such that d{ym,xi) = r with Un{xi) = mi-n.z^x,zn^yUn{z) (which 
exists by the definition of r). By (j23p we have 

^^(xi) = min Un{z)> Un{ym) + 25- 2X1- X2. 

z^XjZ'^y 
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Furthermore (|23p also implies 

Un+iiy) > Un{y) - h > min - A3 > n„(ym) + 2(5 - 2Ai - A2 - A3. 

This proves the claim with yi = ym- D 

Proof of Lemma \2. 7| By part (ii) of Lemma 12.11 it is enough to prove the 
claim for an arbitrary shift of the payoff function /, so by the definition of 
Cf, we can assume that cj = (see Remark II. 13p . 

By Lemma 12. 6| for a given A3 > choose no large enough so that 
ll^^n — 'Un-ill < A3, for all n > uq. Assume that the sequence (n„) is not 
equicontinuous. Then there is a (5o > such that for any p > there are 
infinitely many integers k satisfying osc(u„(,+fc, /)) > Jg. Fix such a k and 
p and define 61 = osc(n„„4.fc, /)). Since — Uno+fll 1^ {k — i)X3, for all 

< £ < k we get that 

osc(n„o+^, p)<Si + 2{k - i)X3. (24) 

Now fix an arbitrary r and let xq and yo be points that satisfy d{xo,yo) < p 
and 

Uno+kixo) - Uno+kiVo) > ^1 - T. (25) 

Applying Lemma [2.81 to the pair (xo,yo) with 5 = di — t, X2 = 5i + 2A3, 
Ai = osc(/, p) and A3 defined as before we obtain points xi and yi such that 
d{xi,yi) < p, and 

UnQ+k-l{xi) - Uno+k-l{yi) >Si-2t- 2Ai - 2A3, 

and 

Uno+k{yo) - Uno+k~i{yi) > di-2T - 2Ai - 3A3. 
Using (p4j) and applying the same arguments inductively, for Ai, A3 and r 
small enough, we obtain a sequence of points (xi), 1 < i < k, which satisfy 
the inequalities 

Uno+k-dxe) - Uno+k-e{ye) > Si - agr - bAi - QA3, (26) 

and 

Uno+k-e+i{ye-i) - Uno+k-e{ye) > Si - agr - bAi - (q + 1)A3, (27) 

where the coefficients satisfy oi = 2, 61 = 2, ci = 2 and 

ae+i = 2ae, be+i = 2(5^ + 1), q+i = 2(q + ^ + 1). 

This leads to 0^ = 2^6^ = 2^+^ - 2 and a = 2^+^ -2^-4. Summing ([27]) 
with these values of coefficients for 1 < £ < k we obtain 

(yo) - > kSi - 2\2t + 4Ai + 8A3). (28) 

Now taking k to be the largest integer not larger than ^°g('^i/(^^^'^-^i+^''^3)) 
and increasing the value of uq if necessary, leads to 

( \ ( r M'^i/(2T + 4Ai + 8A3)) \ 

Uno+k{y0) - Unoiyk) > Sl\^ 2j. (29) 
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Since the values of r, Ai and A3 can be chosen arbitrarily small and 61 is 
bounded from below by So, the right hand side of (129p can be arbitrarily 
large. This is a contradiction with the assumption that c/ = 0. 

□ 

Proof of Theorem \1.7\ By Theorem 1 1.61 (^n) is a bounded sequence of func- 
tions In the case of finite graphs with loops the statement follows from 
Proposition 12. 4i For the case of adjacency graphs, note that, by Lemma 
12.71 (un) is also equicontinuous and, by the Arzela-Ascoli theorem, it has a 
convergent subsequence. Now the claim follows from Proposition 12. 4i □ 

Proof of Theorem \1.8\ in the case of adjacency graphs. If u is a solution to 
([3|) then playing the game with terminal payoff uq = u and running payoff / 
gives the constant sequence of game values u„ = u. In the other direction, 
it is clear that the limit in Theorem 1 1.71 is a solution to the equation □ 

Example 2.9. Let G be a bipartite graph with partition of the vertex set 
into Vi and V2 (meaning Vi n V2 = 0) ^ = ^1 U V2 and all edges in the graph 
are connecting vertices in Vi and V2). Let / be a function on V having 
value 1 on Vi and —1 on ¥2- Then if uq = it is easy to check from ([2]) that 
Un = f n is odd and u„ = if / is even, and therefore the sequence (u„) 
does not converge. However u = //2 is a solution to ([3]). 

From the proof of Lemma 12.61 we can extract the following result about 
the speed of convergence. 

Proposition 2.10. There is a universal constant C > such that, under 
the assumptions of Theorem for n > 2 we have 

AC 

\\Un+l - Un\\ < , (30) 

logn 

where A = (maxuo — minno) + diam(G)^(max / — min/). 
Proof. Again, it is enough to prove the claim when ||n„+i — is replaced 
by Mn(no) = max(u„ — Un-i)- If maxum < minn„ for some m < n then by 
Lemma 12.31 we have u„^^j^(^n-m) ~ — fe(niinn„ — maxum.) for all A; > 0, 
which contradicts the boundedness of (n„) (which in turn follows from the 
assumption c/ = 0). Similarly we get the contradiction when maxn^ < 
minii„ for some n < m. Therefore we have minti„ < maxum for all m and 
n and Lemma 12.21 implies that 

maxun+k — minun < 2A. (31) 

By Lemmas 12. 31 and 12. 61 we know that (M/(no)) is a non-increasing sequence 
converging to 0. For given r > 6 assume n and k are such that for all 
n < m < n + k we have r — 6 < Mm{uQ) < r. Now (fTSl) implies that 

vnaxun+k ~ minii„ > kr — k6 — 2^6. 
Combining this with (131 p we see that, if K{r — 6, r) is the number of indices 
m such that r — 6 < Mm(uo) ^ 1^, then for all integers < k < K{r — 6,r) 
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we have kr - 2^^+^ < 2A. Taking S = r2~^'^/^'~^ we get that K{r - 6,r) < 
1 + 2A/r. Now let tq > and define the sequence r„_|_i = r„(l — 2"^"^/''""^), 
which is clearly decreasing and converging to 0. By the above discussion we 
have 

2A 

K{rn+i,rn)<— + l. (32) 
rn 

Furthermore define N{a,f3) = ^ i^(r„_|_i, r„), where the sum is taken over 
all indices n for which the interval [r„_|_i,r„] intersects the interval [a, /?]. 

Defining s„ = log(2^/r„) we have s„+i = s„ — log ^1 — 2"*^°""^^. Since 

the function s i— >• log ^1 — 2""^°"^^ is negative and increasing, the number of 

indices n such that the interval intersects a given interval [a,h] is 

no more than 

^""^ +2<(6-a)2^''+2 + 2, 



log ( 1 - 2-e'-2 



where we used the inequality log(l — x) < —x, for < x < 1. This together 
with (|32p implies 



A^(2Ae-^ 2Ae-'') < (^(6 - a)2^ + 2j(e'' + l \ 
Therefore we have 

N{2Ae-^,2A) < (4 + o(l))2^*e*, 

and since m({uo) < 2 A, there are no more than (4+o(l))2yl22^/V-i indices 
n such that M/(no) > r, which then easily implies the claim. □ 

Remark 2.11. From Lemma l2.ll (ii) it is clear that removing the assump- 
tion Cf = from the statements of Lemma 12.61 and Proposition 12.101 yields 
lim„(u„+i - Un) = Cf and |||ii„+i - u„\\ - c/| < respectively. 

One of the obstacles to faster convergence is the fact that for each vertex 
X the locations where the maximum and the minimum values of among 
its neighbors are attained depends on n. However, in the case of finite 
graphs with loops, these locations will eventually be "stabilized", if (for 
example) the limiting function is one-to-one. Therefore after a certain (and 
possibly very large) number of steps, we will essentially see a convergence 
of a certain Markov chain, which is exponentially fast. To prove this in 
the next theorem recall some basic facts about finite Markov chains. A 
time homogeneous Markov chain X on a finite state space is given by its 
transition probabilities P{i,j) = f{Xi = j\XQ = i). Denote the transition 
probabilities in k steps as P^{i,j) = P(Xfc = j\Xo = i) (these are just 
entries of the A;th power of the matrix {P{i,j))ij). An essential class of 
a Markov chain is a maximal subset of the state space with the property 
that for any two elements i and j from this set there is an integer k such 
that P''{i,j) > 0. An essential class is called aperiodic if it contains an 
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element i such that the greatest common divisor of integers k satisfying 
P''{i,i) > is 1. The state space can be decomposed into several disjoint 
essential classes and a set of elements i which are not contained in any 
essential class and which necessarily satisfy P^{i,j) > for some integer k 
and some element j contained in an essential class. If all essential classes 
of a Markov chain are aperiodic then the distribution of converges to 
a stationary distribution and, moreover this convergence is exponentially 
fast. This result is perhaps more standard when the chain is irreducible 
(the whole state space is one essential class). However the more general 
version we stated is a straightforward consequence of this special case after 
we observe that the restriction of a Markov chain to an aperiodic essential 
class is an irreducible Markov chain, and that for any element i not contained 
in any essential class, conditioned on Xq = i, the time of the first entry to 
an essential class is stochastically dominated from above by a geometric 
random variable. For more on this topic see [7J. 

Proposition 2.12. Let G be a finite graph with a loop at each vertex, f a 
function on the set of vertices and (un) a sequence of game values played 
with running payoff f . Assuming Cf = 0, let u be the limit of the sequence 
(un) and assume that for each vertex x £ V there are unique neighbors ym 
and yM of x, such that u{ym) = niinjy^^u(y) and u{yM) = maxy^a; t4(?/). 
Then there are constants C > and < a < 1 ( depending on G, f and uq ) 
such that \\un — u\\ < Ca^. 

Proof. Let A = {axy) be the matrix such that Oxy = 1/2 if either u[y) = 
max^^x u{z) or u{y) = min^^^ and otherwise. The Markov process 
on the vertex set, with the transition matrix A, has the property that all 
essential classes are aperiodic. To see this fix an essential class I C F let x be 
a vertex such that u{x) = max/ u, and observe that a^x = 1/2. Therefore the 
distribution of converges exponentially fast to a stationary distribution. 

Since u = lim„ ii„, there is an no such that for n > no and any vertex x the 
unique neighbors of x where u attains the value max^^a;ti(z) {umizr^x u{z)) 
and where Un attains the value max^^^; (min^^a; are equal. 

Writing functions as column vectors, this means that Un+i = Aun + / for 
n > Uq. Thus, defining w„ = Un+i — Un, for n > no we have 

Vn+l = Un+2 " Un+l = AUn+l - AUn = AVn- 

This means that for any /c > we have Vn(,+k{x) = "E^xivnoi^k))- Therefore 
the sequence of functions {vnQ+k)k converges exponentially fast. Since we 
necessarily have lim„f;„ = the claim follows from — n|| < 'Yl^=n ll^fcll- 

□ 

Our next goal is to prove Theorem 11.81 for all finite graphs. Recall the 
(nonlinear) operator Af from Lemma 12.31 For a real number c G M, and 
a function u define Df{u,c) = ||^j_cU — u\\. To prove the existence of 
a solution it is enough to prove that Df has a minimum value equal to 0. 
First we use a compactness argument to prove that it really has a minimum. 
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For the rest of this section all the graphs will be arbitrary connected finite 
graphs. 

Lemma 2.13. Let G be a finite connected graph, and f and u functions on 
V. Then 

maxn-minu < 2'i'^'^(^)+i(||/|| + L>j(n, 0)). (33) 

Proof. Assume that the function u attains its minimum and maximum at 
vertices Xm and xm respectively. Let Xm = Ho, Hi, ■ ■ ■ ,yk-i,yk = xm be a 
path connecting Xm and xm with k < diam(G). Observe that 

Afu{yi) > + f{yi), 

for i = 0, . . . ,k — 1. Estimating the left hand side of the above equations by 
AfU < u + Df{u,0) we get 

n(2/,+i) < 2u{yi) + 2Df{u, 0) - 2/(y,) - u{x^), 

for i = 0, — 1. Multiplying the i-th. inequality by 2^~^~^, for i = 

0, . . . k — 1 and adding them we obtain 

u{xm) - u{xn,) < (2'^+! - 2)(D/(n,0) - min/), 

which implies the claim. □ 

Lemma 2.14. Under the assumptions of Lemma \2.13\ the function Df{u,c) 
has a minimum value. 

Proof. Since is a continuous function, we only need to prove that r : = 
inf Df = miuxi Df, where the right hand side is the infimum of the values 
oi Df over W x / for a bounded set of functions U and a bounded interval /. 
First assume that c is a constant large enough so that f + c has all values 
larger than r + 1. If vertex where a function u attains its minimum, 

we have 

^(max u{y) + min u{y)) + f{x) + c > u{xm) + r + 1. 

This implies that Df{u, — c) > r + 1 for any function u. Similarly for suffi- 
ciently large c we have that Df{u,c) > r + 1 for any function u. Therefore 
there is a bounded interval I such that the infimum of values of D{u, c) over 
all functions u and c ^ / is strictly bigger than r. 

Furthermore by Lemma 12.131 we can find a constant K such that for any 
c £ / we have that maxu — minu > K implies Df{u,c) > r + 1. Also since 
Df{u + A, c) = Df{u, c) for any A G M, we have that r = inf^^x/ Df where U 
is the set of functions such that minu = and maxu < K. Since the set lA 
is bounded the claim follows. □ 

Proof of Theorem \1.8\ in the case of finite graphs. Assuming the existence of 
a solution the argument proceeds as in the proof of the adjacency case. 
By the same argument, to show the other direction, it is enough to prove 
that there is a constant c for which there is a solution to ([3]), when the 



20 



T. ANTUNOVIC, Y. PERES, S. SHEFFIELD, AND S. SOMERSILLE 



right hand side / is replaced by / — c, since then we necessarily have 
c = 0. In other words it is enough to show that min Df = 0. By Lemma 
12.141 this minimum is achieved and denote it by m = minDf. Assume 
that m > 0. Fix a pair (u, c) where the minimum is achieved and define 
^u,c •= • Af^cuix)—u{x) = m}, := {x : Af-cu{x) — u{x) = —m} and 
Su,c '■= U S~^. By definition Su,c 7^ 0- If S:^^. = then there is a 5 > 
small enough so that Af_c+5U — u < m, and of course Af_c+iU — u > —m. 
This implies that Df{u,c — 6) < m, which is a contradiction with the as- 
sumption that m = mini?. Therefore S^^ ^ 0, and similarly ^ 0. 

Call a set Sr C S^^ removable for function u, if both of the following two 
conditions hold: 

(i) For every x £ S^^ there is a y ^ Sr so that y ~ x and u{y) = 
mm;,^^ u{z). 

(ii) There are no x G 5"^^ and y £ Sr so that y ~ x and u{y) = 
max^^^ u{z). 

By increasing values of the function u on Sr we can remove this set from S^^. 
More precisely, define the function us so that us{x) = u{x) for x ^ Sr and 
us{x) = u{x) + (5 for X G 5^. Since the graph G is finite, for 6 small enough 
and all points x ^ S^^, we have Af_cUs{x) — us(x) < m. Furthermore, by 
the above two conditions, if 5 small enough, for any point x E S^^ we have 
Aj_cM5(x) = Af^cu{x). On the other hand for x € Sr we have 

Af^cUsix) -us{x) = m- (5, 

and therefore S^^ ^ = S^^^Sr- Moreover S^^ ^ C 5"^ is obvious. 

Similarly we can define removable sets S~ contained in S~^ so that there 
are no x G 5"^ and y G Sr such that u{y) = min^^^: u{z) and that for every 
X G S~^ there is a y ^ Sr such that u{y) = maxz^x uiz) ■ This set can be 
removed from S~^ be decreasing the value of u on this set. Note that the 
removable sets in S^^ and can be removed simultaneously as described 
above. Thus if a pair {u,c) minimizes the value of Df, and u is obtained 
from u by removing removable sets in and 5"^, then the pair (n, c) also 
minimizes the value of Df, and moreover Su^c C Su,c- 

Call a function u tight (for /) if there is c G M such that the pair (n, c) 
minimizes Df, and so that the set Su,c is of smallest cardinality, among all 
minimizers of Df. By the discussion above, tight functions have no non- 
empty removable sets. For a tight function u define v = Af^cU- By Lemma 
12.31 we have that Df{v,c) = \\Af-cV — v\\ <m and because m = vaiaDf we 
have Df{v,c) = m. 

Now observe that it is enough to prove that for any tight function u and 
V = Af-cU, the set S^^\S^^ is removable for function v. To see this first 
note that by symmetry the set S~^\S~^ is also removable for v. Let vi 
be a function obtained by removing all these vertices as described above. 
In particular we have ui(x) = v{x) = u{x) + m for x G S^^ and fi(x) = 
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v{x) = u{x) — mioixG S~^. The function vi then satisfies 5^^^ C and 
Syj^^c ^ 'Su,c- By tightness of u it follows that 5*+ = S'+g and 5*" ^ = 'S'u,c 
and thus the function vi is also tight. Now we can repeat this argument to 
obtain a sequence of tight functions (vk) such that 5*+ = 5^^, S~^^^ = S~^, 
Vk{x) = u{x) + /cm for x G 5"^^ and = u{x) — km for x G 5"^. Since 

D(vk,c) = m for all and linifc (max — minw^) = oo, Lemma l2.13l gives a 
contradiction with the assumption that m > 0. 

Thus it is only left to prove that for any tight function u and v = Aj-cU 
the set S^^\S^^ is removable for function v. For this we need to check 
the conditions (i) and (ii) from the definition of the removable sets. Take 
a vertex x £ and note that since Af_cv{x) — v{x) = max(f — u) and 
V = Af_cU, by Lemma [213] for a yi ~ x such that u{yi) = minz^xuiz), we 
have v{yi) = in.mz:^xv{z) and yi £ S^^ which checks the first assumption. 
Furthermore by Lemma 12.31 for any 7/2 such that ^(^2) = ^axz^xv{z) we 
have 2/2 £ which also checks the second assumption in the definition of 
removable sets. □ 

Next we present two examples for which we explicitly calculate the value 
of the Player Fs long term advantage cj. 

Example 2.15. If G is a complete graph with loops at each vertex and / 
a function on the set of vertices, then cj = (max / + min/)/2. To see this, 
use the fact that c/ defined as above satisfies c/+a = c/ + A for any A G M, 
and that c/ = implies that u = f solves (j3]). 

When G is a complete graph without loops the situation becomes more 
complicated. If the function / attains both the maximum and the minimum 
values at more than one vertex then again we have cj = (max / + min /)/2, 
and again in the case max /+ min / = the function u = f satisfies equation 
0. _ 

If the maximum and the minimum values of / are attained at unique 
vertices then 

max/ + min/ max2 / + min2 / 
Cf = + , (34) 

where max2 / and min2 / denote the second largest and the second smallest 
values of the function / respectively. To prove this assume the expression 
in (fM|) is equal to zero, and let xm and Xm be the vertices where / attains 
the maximum and the minimum value. Then define a function u so that 
u{xm) = (2max/ + max2 /)/3, u{xm) = (2 min / + min2 /)/3 and u{x) = 
/(x), for X ^ {xm,XM}- Now using the fact that c/ = and that u attains 
its maximum and minimum values only at xm and Xm respectively, it can 
be checked that u solves ([3]). 

Finally in the case when the maximum value of the function / is attained 
at a unique vertex xm and the minimum at more than one vertex we have 
Cf = {2 max / + max2 / + 3 min /) /6. When this expression is equal to zero, 
one solution u of the equation (j3]) is given by u{x) = f{x) for x 7^ xm 
and u{xm) = (2 max / + max2 /)/3. Similarly when the maximum of / is 
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attained at more than one vertex and minimum at a unique vertex we have 
Cf = (2min/ + min2 / + 3max/)/6. 

Example 2.16. Consider a hnear graph of length n with loops at every 
vertex, that is take V = {1, . . . ,n} and connect two vertices if they are at 
Euclidean distance or 1. Let / be a non-decreasing function on the set of 
vertices, that is f{i) < f{i + 1), for 1 < i < n — 1. By induction and ([2]), 
running the game with the vanishing terminal payoff and the running payoff 
/ gives sequence of game values (un), each of which is a non-decreasing 
function. Representing functions column vectors, we have Un+i = 

Aun + /, where A = {aij) is a matrix with an = ann = 1/2, Oij = 1/2, if 
|z— j| = 1 and aij = for all other values of i and j. Therefore f„ = Un+i—Un 
satisfies Vn+i = Avn- Using this we see that Vn{x) = Kx{f{Xn)), where Xn is 
the simple random walk on the graph with the vertex set {1, . . . , n} where i 
and j are connected with an edge if \i—j\ = 1 and with loops at 1 and n. The 
stationary distribution of the random walk is uniform on {1, . . . ,n}, 

and this is the limit of the distributions of X„ as n tends to infinity. From 

here it is clear that lim„t;„(j;) = ^ X^iLi /(^)) /"- equal to the 

average of the values of function /. 

The condition that / is monotone is necessary. Consider for example the 
linear graph with loops and three vertices and the function (/(I), /(2), /(3)) = 
(— 1, 2, —1). Then by ([2]) we have tio = 0, ui = / and U2 = / + 1/2 which 
implies that Un+i = + 1/2 for all n > 1 and by Lemma l2. II (i) we have 
Cf = 1/2. 

3. The continuous case 

The main goal of this section is to study the game values on Euclidean 
e-adjacency graphs, as defined in the Section 1, to obtain the existence of 
viscosity solutions to the equation ([5]). One of the main concerns will be the 
dependence of the game values and limits, obtained in the previous section, 
on values of step sizes e. The following example shows that the issue starts 
already with the Player I's long term advantage c/(e) (recall that c/(e) was 
defined as the Player I's long term advantage for a game played on an e- 
adjacency graph with the running payoff /). 

Example 3.1. This example shows that, in general, for Euclidean e-adjacency 
graphs on a domain and a continuous function / : 17 — )• M, the value of 
c/(e) depends on e. First observe a trivial fact that for any 17 of diame- 
ter diam(r2) and / we have c/(diam(ll)) = (max / + min/)/2. Next let 
O = (0, 1) and let / be a piecewise linear function that is linear on the inter- 
vals [0, 1/2] and [1/2, 1] and has values /(O) = /(1/2) = 1 and /(I) = -1. 
By the above observation we have c/(l) = 0. However notice that from 
([2]) it is clear that, for any e, playing the game with step size e, the van- 
ishing terminal payoff and the running payoff /, the game values will be 
non- increasing functions on [0, 1]. Therefore in the game of step size 1/2 the 
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game values Un at points 0, 1/2 and 1 are equal to the game values played on 
the linear graph with three vertices and loops on each vertex, with terminal 
payoff zero and running payoff equal to 1, 1 and —1 at the leftmost, central 
and the rightmost vertex respectively. Using Example 12.161 and going back 
to the game on [0, 1] this implies that c/(l/2) = 1/3. 

For a more comprehensive example, construct a monotone function / on 
[0,1] such that no value of (2" + l)""*^ ^^^=0 /(^^~") attained for two 
distinct integers n. By the above reasoning and Example 12.161 the value of 
c/(e) varies for arbitrarily small values of e. 

For the remainder of this paper, all the graphs are assumed to be e- 
adjacency graphs and the dependence on e will be explicitly specified. 

Theorem 11.91 settles the issue raised in the above example. We will need 
several technical lemmas for the proof of Theorem 11.91 The main ingredient 
of the proof is a comparison between the values of discrete infinity Laplacian 
with different step sizes from Lemma 13.41 The idea for (as well as one part 
of) this lemma came from [T]. 

Lemma 3.2. /// and u are continuous functions on a compact length space 
V such that — A^u < / then Cf{e) > 0. Similarly — A^u > / implies 
Cf{e) < 0. 

Proof. The second claim follows by replacing u and / by —u and — / respec- 
tively, so it is enough to prove the first one. The condition —A^u < f can 
be rewritten as 

1 / \ 

— I maxu(z) + minn(z) I H f{x) > u{x). 

2 \ z^x z^x J 2 

Thus the game value u\ of the first step of the game, played with the terminal 
payoff = u, running payoff e^//2 and step sizes e satisfies ui > uq. By 
Lemma 12.31 we have Un+i > Un for any n, hence c/(e) > is clear. □ 

Lemma 3.3. Mapping e i— )■ c/(e) is continuous on M"*". 

Proof. For a given e let S C{V) be a solution of — A^n^ = / — c/(e), 
which exists by Theorem 11.81 and Remarks 11.31 and 11.131 Since for any 
u £ C{V), it holds that e i— )• — A^n is a continuous function from M+ to 
{C{V), II • lloo), so for a fixed e and any 5 > we can find r/ > such that 
I — A^Wel < / — c/(e) + 6 whenever |ei — e| < tj. Now by applying Lemma 
13.21 we see that for such ei we have |cj(ei) — cj(e)| < 5, which gives the 
continuity. □ 

As mentioned above, the main part of the proof of Theorem 11.91 is con- 
tained in the following lemma. For Euclidean e-adjacency graphs, the first 
inequality in (I35p already appeared as Lemma 4.1 in [T]. However since 
their definition of the discrete Laplacian was somewhat different close to the 
boundary dQ, their estimates held only away from dQ. This issue does not 
appear in our case and their proof goes verbatim. For reader's convenience 
we repeat their proof of the first inequality in (i35]l . 
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For a function u: V ^ M we first define 17* = max^^^i^a. u{z) and = 
™iiizGB{a;,e) u{z). Furtliermore define T+u(x) = u'^{x) — u{x) and T~u{x) = 
u{x) — u^{x) (this corresponds to eS^ and eS~ in [Ij). Now we can write 
-A^n = (r-n-r+n)/e2. 

Lemma 3.4. Suppose that u G C(y) satisfies —A^u < fi for some fi G 
C{V). Then we have 

- A^^lf < t"' and - Al^Tf < A^ (35) 

If, in addition, we have — A^u < /2 for some f2 G C{V) then 

-A^r <(8^'^+D/9. (36) 

Proof. In the proof we will repeatedly use the following arguments. If 
zq, Zi ^ V are such that zi G B{zq,5) and v{zi) = v^{zo) then we have 

Ts^v{zo) = v(zi) - v{zo) < T^v{zi). (37) 

Furthermore the assumption —A^v < f implies that 

T+r;(zo) < Tiv{z,) < T+v{zi) + 6^f{z,). (38) 

Denote points yi G B{x,e), y2 G B{yi,e), zm ^ B{x,2e) and z^ G 
B{x, 2e) so that u{yi) = lf{x), u{y2) = u^(yi), u{zm) = u^'^(x) and u(zm) = 
U2^{x). We calculate 

T+lf{x) = u^'ix)-lf{x) 

= (u^'ix) - u{y2)) + {u{y2) - n(yi)) 
>T+uiy2) + T+uiyi) 

>2r+n(yi)-e2/i(y2) 
>2T+u{x)-e\f,{y2)+2fi{yi)) 

> T+u{x) + T-u{x) - e\fi{y2) + 2/i(yi) + f^{x)). (39) 

In the first inequality we used the fact that B{y2,€) C i3(x,3e) and in the 
second inequality we used ([38]) with zq = yi, zi = y2 and 6 = e. In the next 
line we again used (|38p with zq = x, z\ = y\ and = e, and in the last line 
the assumption — A^u < f\. 
Furthermore we have 

T2ji'{x) =^{x) — min u*^ 

i3(a;,2€) 

< (u'^(a;) — n(x)) + (n(x) — Ufix^) 

= T+u(x) +r-n(x). (40) 

The inequality above follows from the fact that for every z G B{x,2e) we 
have max^(2 u > mmB{x,e) u. Now the first inequality in (|35]) is obtained 
by subtracting (j39|) from (jiOj) . 
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For the second inequality in ((35]) note that 

T+r (x) = v^'ix) - u{yi) > r+n(yi), 

and 

T^Jf{x) = lf{x) — min If < u{yi) — u{x) < T~u[yi). 
B{x,t) 

Subtracting the above inequahties it follows that —A^^ti^{x) < —A^u{yi) < 

h{yi)<Ti{x). 

We prove the inequality (|36p similarly. First we calculate 

T+tf{x) = v^^{x)-if{x) 

= (^'(a;) - u{zm)) + (u{zm) - u{y2)) + {u{y2) - u{yi)) 

> T^^u{zm) + T^u{yi) 

> T+ui^M) + T-u{yi) - e2/i(yi) 
>T+uizM) + T+u{x)-e''fi{yi). 

In the third line we used the fact that y2 G B{x,2e) which implies that 
u{y2) < u{zm), in the fourth line the assumption and in the last line ([37|) . 
Using similar arguments again we have 

TZlf(x) =lf(x) — min if 

^ B{x,3e) 

< tf{x) -U2^{x) 

= {u{yi) - u{x)) + (n(x) - u{zm)) 
= T^u{x) + r2~u(x) 
<T^u{x)+T+u{x) + {2eff2{x) 

< T+u{x) + T^^u{zm) + {2eff2{x). 

Now subtracting the above calculations and dividing by (3e)^, we obtain 

-A^lf{x) < (T^M^m) - T+u{zM))/{'d€^) + 4/2(x)/9 + /i(2/i)/9, 

from where the fact follows directly. □ 

Recah the notation osc(/,(5) = suprf(^^y)<5 - f{y)\. 

Proposition 3.5. For any f G C{V), e > and any positive integer n we 
have 

max{|c/(e2-") - c/(e)|, |c/(e3-") - c/(e)|} < osc(/, 2e). (41) 

Proof. First note that the functions and f ^ differ from / by at most 
osc(/, r) at any point, which easily implies 

max{|c/(/9) -cjr(p)|,|c/(p) -c/^(p)|} < osc(/,r), (42) 

for any p. 

Taking n to be a continuous function such that — A^n = f — Cf{5), by 
Lemma [3.41 we have —A'^u^ < f — Cf{S) and therefore also — A^u^'^ < 
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f^^ — Cf{S). By Lemma 13.21 these inequalities and their symmetric counter- 
parts imply that 

c/^^(2(5) <Cf{6) < Cj2s{26) and < Cf{5) < Cjis{35). 

Applying these estimates inductively to 6 = 62"", . . . e/2 and 5 = eS"", . . . e/3 
respectively, we see that 

c^Je) < c/(e2-") < c^2.(e) and c^Je) < cj(e3-") < c^2.(e). 
Using (j42p with r = 2e and p = 2e, these inequalities imply (|4ip . 

□ 

Proof of Theorem \1.9l Since min/ < Cf{e) < max/, there are accumulation 
points of Cf{e) as e | 0, and we only need to prove that there is only one. 
Suppose that there are two such accumulation points ci < C2 and denote 
5 = C2 — ci. Let Ii and I2 be disjoint open intervals of length (5/2, centered 
around ci and C2 respectively. Let eo be a positive real number such that 
osc(/, eo) < 6/4, and consider the open sets Ji and J2 defined as Jj = 
cj^ih) n (0,eo/2). First note that the set {2-"'3" : m,n £ Z+} is dense in 
M^. This follows from the fact that {nlogS — mlog2 : m,n £ Z^} is dense 
in M, which in turn follows from the fact that log 3/ log 2 is an irrational 
number. Take an arbitrary t £ Ji and, since {s/t : s G J2} is an open set in 
M"*", we can find non- negative integers mo and no such that 3"'°2~"^^t G J2. 
Therefore 

|c/(t) -c/(3"°2-™°t)| > 5/2. 

However this gives a contradiction, since both t and 3"°2~'"''t lie in the 
interval (0,eo/2), and so by Proposition 13.51 we have 

max{|c/(2-™«t) - c/(t)|, |c/(3"»2— «t) - c/(2-™«t)|} < osc(/, eo) < S/A. 

□ 

Proposition 3.6. For a sequence (e„) converging to zero, let be a 

sequence of continuous functions on a compact length space V, satisfying 
— A^n„ = / — c(e„). Then zs an equicontinuous sequence and for all 
n large enough we have 

maxii„ — minn„ < 6diam(l/)^||/||. 

Furthermore, any subsequential limit of the sequence (un) is Lipshitz con- 
tinuous, with the Lipshitz constant 5diam(y)||/||. 

Proof. It is enough to prove that for n large enough and any x G V, we have 
that 

<5diam(y)||/||e„. (43) 

This is because, for any two points x,y G V and n such that e„ < d{x, y) 
there are points x = xo,xi, . . . , Xk,Xk+i = ymV such that d{xi,Xi+i) < 
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and k = [d{x,y)/en\- Assuming that (jl3]l holds we have that 

k k 
Un{y) - Un{x) = ^{un{xi+i) - Un{xi)) < ^T+n„(2;j) < Kd{x,y) + Ken, 
i=0 i=0 

(44) 

where K = 5 diam(y)||/||. On the other hand for e„ > d{x,y) we have 
Un{y) — Un{x) < Ken- These two facts then easily imply the equicontinuity, 
the required bound on maxti„ — minu^, for n large enough and the Lipshitz 
continuity of subsequential limits. 

The rest of the proof will be devoted to establishing the bound in ()43p . 
We will use the "marching argument" of Armstrong and Smart from Lemma 
3.9 in [Ij. First by ([38]) if y G B{x,en) is such that Un{y) = u^"(x) then 
using the fact that min/ < Cf{en) < max / we have 

r+n„(x) < Tlun{y) + elWf - Cf{en)\\ < T+Uniy) + 26^ ||/||. (45) 

For a fixed n let xq £ V he a point where the value of T^Un is maximized 
(it's a continuous function so it can be maximized) and let M„ = T^Un{xo) 
be the maximal value. Using the same argument as in (j44p and the fact that 
V is bounded we have 

Un{y) - Unix) < + l)Mn. (46) 

Then for any k let x^+i G B{xk,en) be such that Un{xk+i) = u^'^"{xk)- By 
(gSD we have that T+Unixk+i) > T+Un{xk) -2en'^\\f\\ and thus r+ti„(xfc) > 
T+ti„(xo) — 2A:e^||/|| which implies that for any m > 1 

m—l 

Un{xm) - Un{xo) = ^ T+n„(3;fc) > mT+n„(xo) - m^e^| 

k=0 

Combining this with (j46p we obtain that 

mMn-mhl\\f\\<(^^^^ + l)Mn, 



which gives 



M„ < 



vr?e\ 



m—l — diam(y)/e„ 
Plugging in m = [2diam(y)/e„ + 2j proves (j43p for e„ small enough. □ 

Proof of Theorem \1.1(K The claim follows directly from Proposition 13.61 us- 
ing the Arzela-Ascoli theorem. □ 

Finally Theorem 13.91 below proves Theorem II. Ill However we will first 
need to state an auxiliary result which appeared as Lemma 4.2 in [Ij. For 
X £ and e > 0, define B(a;,e) as the closed ball around x of Euclidean 
radius e. Also define the discrete infinity Laplacian on the whole as 



A^,f (x) = -?r f max v + min v — 2v{x) 

e2 \B(x,e) B(x,e) 
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The first part of tlie following lemma is the content of Lemma 4.2 in pLj, 
while the second part is contained in its proof (see (4.5) in [Ij). 

Lemma 3.7 (Lemma 4.2 and (4.5) from [1]). For any open set U , function 
if G C^{U) and eo > there is a constant C > 0, depending only on ip, such 
that the following holds. 

(i) For any point x £ U that satisfies B(x, 2eo) ^ U and Vip{x) ^ we 
have 

-A^ifix) < -A^v'(x) + C(l + \V^{x)\-')e, 
for all < e < eo. 

(ii) For any < e < eo if ^ = V ^{x) /\V (p{x)\ and w G B(0, 1) is such 
that ip{x + ew) = maxB(x,e) '^j then |v — w| < C\V Lp{x)\^'^e. 

Next we give an auxiliary Lemma needed for the proof of Theorem 13.91 
First define a cone in M"^ with vertex x, direction v G W^, |v| = 1, angle 
2 arcsin a and radius r as 

C(j;, v,a,r) = |Aw G M'' : < A < r, |w| = 1, w • v > 1 - a|. 

Lemma 3.8. Let Q. be a domain with boundary dQ, let xq G 50. Assume 
If G C°°(0) is a smooth function on the closure of Q and Vuf{xo) > 0. 
Then we can find positive a and r and an open set U containing xq such 
that C{x,-Vip{x)/\V(p{x)\,a,r) cTl, for all x £ U nil. 

Proof. Denote v = Vip{xo)/\V(p{xo)\. By the continuity of Vip it is enough 
to prove that for some a and r and U we have Cx = C(x,— v, a,r) C Q, 
for all a; G C/ n O. Moreover define the reverse cone = C(x,v, Q,r). It 
is clear that we can find a and r satisfying Cx-q C O. Moreover, since the 
boundary dQ is C^, it is easy to see that, by decreasing a and r if necessary, 
we can assume that Cx C and C O^, for all x G with — xq] < 2r. 
Then if x G is such that |x — xo| < r and Cx {Z! we can find a point 
y G 50 n Cx- Then the fact that x G Cy n and |y — xo| < 2r leads to 
contradiction. □ 

Theorem 3.9. Let C M*^ 6e a domain with boundary, (e^) a sequence 
of positive real numbers converging to zero and (u„) a sequence of continuous 
functions on 0, satisfying — A^ti„ = / — c/(e„). Any limit u of a subsequence 
of (un) is a viscosity solution to 

( - AooU = f - Cf inO,, 

\ VyU = on do.. 

Proof. We denote the subsequence again by (u„). To prove that n is a 
solution to ([5]) we will check the assumptions from Definition 11.41 for local 
maxima. The conditions for local minima follow by replacing u and / by 
—u and — /. Let G C°°(0) be a smooth function and xq G be a point 
at which u — ip has a strict local maximum. We will prove the claim for 
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xq E 50. For the case xq £ ^ see either of the two proofs of Theorem 2.11 
in [l] (in Sections 4 and 5). 

Assume that Vu^{xo) > 0. For k large enough we can find points x^ so 
that hmfc Xfc = xq and such that Uk — has a local maximum at Xk- We 
can assume that for all k we have |V(/3(xfc)| > c, for some c > 0. Denote 
Vfc = —Vip{xk)/\Vip{xk)\ and for a given A; large enough, a point G 
B(2;fc,efc) such that ip{xk) = min-Qf^^^ ,:^-^ (p. By Lemma [3^1 (ii) and Lemma 
13.81 we can find a and r such that for k large enough we necessarily have 
Xk G C(xfc,Vfc,a, r) C il. For such k this readily implies that 

- A^^ixk) < -A^(/7(xfc). (47) 

Lemma 13.71 further yields that there is a constant C such that, for k large 
enough 

- A^^ixk) < -A^(^(xfc) + C(l + c-i)efc. (48) 
Plugging into (jiSj) we obtain 

- AooV5(xfe) < -A^v?(xfc) + C(l + c-i)efc. (49) 
Since Uk — ^ has a local maximum Xk we have that 

-A^Vj(xfe) < -A^Ufc(a;fc) = /(a^fc) - c/(efc). 

Inserting this into (j49p and taking the limit as /c tends to infinity implies 
that -Aoo^pixo) < f{xo) -Cf. □ 

Proof of Theorem The claim follows directly from Theorems 11.101 and 
13:91 □ 

To prove Theorem 11.121 we will use Theorem 2.2 from [2]. A general 
assumption in [2] is that the boundary d^l is decomposed into two disjoint 
parts, r^) 7^ on which Dirichlet boundary conditions are given and F^v 
on which vanishing Neumann boundary conditions are given. While the 
assumption F^ 7^ is crucial for their existence result (Theorem 2.4 in [2J) 
this assumption is not used in Theorem 2.2 from p]. In the case when 
F/) = their result can be stated as follows. 

Theorem 3.10 (Case To / of Theorem 2.2 in [2J). Let Q, be a convex 
domain and f,u: Q — t- R continuous functions such that u is a viscosity 
subsolution to the equation ([5]). Then for any e > it holds that —A'^u^ < 

f ^ on 0,. 

Proof of Theorem By Theorem 11.111 the equation (j5]) has a solution 
when c/ = 0. Now assume that -u is a viscosity solution to ([5]). By Theorem 
13.101 we have that —A^u^ < / + osc(/, 2e). Now Lemma 13.21 implies that 
c/(e) > — osc(/, 2e). Similarly one can obtain cj(e) < osc(/, 2e) and the 
claim follows by taking the limit as e J, 0. □ 

Remark 3.11. In the one dimensional case (say = [0,r]) the viscosity 
solutions to the equation ([5]) are standard solutions to the equation —u" = f 
where u'(0) = u'{r) = 0. It is clear that in this case Cf = \ Jq f{x)dx. 



30 



T. ANTUNOVIC, Y. PERES, S. SHEFFIELD, AND S. SOMERSILLE 




Figure 1 . Example of functions from Remark 13.111 Func- 
tions /i, /2, /a and have supports in discs Ui, U2, U3 
and [/4 respectively. The shaded area is the support of the 
function g. 

The fact that c/ is a linear functional of / relies heavily on the fact that 
the infinity Laplacian in one dimension is a linear operator. For higher 
dimensional domains / 1— )• c/ is in general not a linear functional. Next we 
show that a two dimensional disc is an example of a domain on which c/ is 
a nonlinear functional of / (essentially the same argument can be applied 
for balls in any dimension higher than one). 

Take $7 to be a two dimensional disc of radius r centered at the origin, and 
assume that on C(0) the mapping / 1— )• is a linear functional. Since it is 
clearly a positive functional and ci = 1 by Riesz representation theorem it 
is of the form Cf = J-^fdfj,, for some probability measure fi on Q. Let / be a 
radially symmetric function on that is /(x) = ^(Ixl), where g: [0, r] — )• R 
is a continuous function. Let Un- — )• M and Vn : [0, r] — )• R be the sequences 
of game values with the running payoff / played on Q and the running payoff 
g played on [0, r] respectively, both games played with vanishing terminal 
payoff and step size e. Using induction and ([2]) one can see that for any n we 
have Un{x) = Vn{\x\). Thus, using the expression for Cf in the one dimen- 
sional case we have that fj, is necessarily of the form fi(dx, dy) = — "^^"^^ , 

that is /i is a radially symmetric measure which assigns equal measure to 
any annulus of given width. 

Next let Ui, U2, U3 and U4 be disjoint discs with centers at (0,r/2), 
(r-/2,0), (0,— r/2), (— r/2,0) and radii r/4. By Qi denote the smallest disc 
with the center (0, r/2) which contains il.. Let /i : l^i — t- [0, 1] be a function 
with support in Ui and which values are radially symmetric around (0, r/2), 
decreasing with the distance from (0, r/2) and such that fi{x) = 1 whenever 
the distance between x and (0,r/2) is no more than r/4 — 61. Playing the 
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game on Qi with running payoff /i leads to 

2 /■r/4-<5i 1 

-Ch>^ ^dt=^- fi. (50) 

Let and be the sequences of game values with the running payoff /i 
played on and the running payoff /il^y played on 0, respectively, both 
games played with vanishing terminal payoff and step size e. It is clear that 
are radially symmetric functions on Jli with values decreasing with the 
distance from (0, r/2). Using this and the induction on n one can see that 
u^nix) < Wnix) for any x G 17. Therefore it holds that cj-^ < cj^i^y, which 
together with ([50]) implies 

1 _ 25i 
'f^ ~ 6 ~ 'Sr' 

Take /2, /a and /4 to be equal to the function /i rotated clockwise for tt/2, 
TT and 37r/4 respectively and / = X^j^^ -^^ symmetry and the assumed 
linearity, the function / : $7 — t- M satisfies 

Now take g: 17 — )• [0, 1] to be a radially symmetric function on il, such that 
g{x) > if and only if r/4 — 62 < \x\ < 3r/4 + 62 and such that f < g. 
Again by the assumption we have 

1 /■3'-/4+'52 1 2^2 

c,<- / ldt = - + -^. (52) 

Since < c^, for small 5i and ^2, inequalities (jST]) and ([52]) give a con- 
tradiction with the assumed linearity of the mapping / 1— )• cj. See Figure 

m 

4. Uniqueness discussion 

As Figure [2] illustrates, in the graph case, once we are given /, we do not 
always have a unique solution to 

u{x) (minu(y) + maxn(y)) = f{x), 

2 y^x yr^x 

even in the case of a finite graph with self loops. When the running payoff 
at each vertex is /, the corresponding "optimal play" will make u{xk) plus 
the cumulative running payoff a martingale {xk is the position of the token 
at the A;th step). Under such play, the players may spend all of their time 
going back and forth between the two vertices in one of the black-white pairs 
in Figure [2j (In the case of the second function shown, the optimal move 
choices are not unique, and the players may move from one black-white pair 
to another.) The basic idea is that as the players are competing within one 
black-white pair, neither player has a strong incentive to try to move the 
game play to another black-white pair. 
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A continuum analog of this construction appeared in Section 5.3 of [PSSW], 
where it was used to show non- uniqueness of solutions to AooU = g with 
zero boundary conditions. (In this case, g was zero except for one "positive 
bump" and one symmetric "negative bump" . Game players would tug back 
and forth between the two bumps, but neither had the ability to make the 
the game end without giving up significant value.) It is possible that this 
example could be adapted to give an analogous counterexample to unique- 
ness in our setting (i.e., one could have two opposite-sign pairs of bumps 
separated by some distance on a larger domain with free boundary condi- 
tions — and players pulling back and forth within one pair of bumps would 
never have sufficient incentive to move to the other pair of bumps — and 
thus distinct functions such as those shown in Figure [2] could be obtained). 
However, the construction seems rather technical and we will not attempt 
it here. 




Figure 2 . The difference equation ([3]) does not always have 
a unique (up to additive constant) solution. On each of the 
three copies of the same graph shown above (assume self- 
loops are included, though not drawn), we define / to be the 
function which is —1 on black vertices, 1 on white vertices, 
and on gray vertices. Then each of the three functions illus- 
trated by numbers above solves In each case, the value 
at a gray vertex is the average of the largest and smallest 
neighboring values. The value at a black (white) vertex is 
1 less (more) than the average of the largest and smallest 
neighboring values. 
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